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&BSTP&CT 

If Institutions of higher education are serious about 
encouraging excellence in teaching, then they must be prepared to 
reward such excellence in a manner similar to that tor excellence in 
scholarly achievement. The first step to achieve this goal would be 
to establish a standard system of evaluating instructional 
effectiveness that would have as its two major outcomes (a) reward 
for success and (b) guidance for improvement* Evaluation of stuaent 
attitudes about instructors and instruction is perhaps the most 
commonly used method of assessing instructional effectiveness* The 
goality of student learning is another very important variable to 
consider* The student attitude or rating data should be presented to 
the Instructor in an easily readable and interpretable format « 
Faculty (peer) and department head attitudes and judgments should be 
presented to the faculty member in an easily interpretable report* If 
part or all of the assessments of instructional effectiveness are not 
highly positive r then the instructor should determine what needs to 
be done to Impr 'ive the institutional weaknesses* These needs cottld 
then be presentf;>d in the form of a proposal* The higher level 
administration of both the Institution and the faculty should accept 
and encourage the use of such a system as an integral part of the 
institutional reward scheme*. (Author/P6) 
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IMPROVING INSTRUCTIONAL EFFECTIVENESS 
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Wien a graduate student attf^tns the Th.D* degree and prepared to 
embark on his professional career at any on£ of the major universities 
In the United States or abroad* he Is still generally quite impression- 
able* For Instance, when his department head or dean tells him that the 
Institution embraces the three general objectives of excellence Itt 
research, teaching* and service and that rewards are based upon satis- 
factory to excellent performance in any one or a combination of those 
objectives, the new faculty maaber may believe that this Is the case. 
The sad fact Is that after a short period of time at his Institution, 
he realizes that although the three rbjectlves of research, teaching, 
and service are appropriate for any Institution of higher education, most 
of the Institutions reward faculty pr'jnarlly for their performance In the 
research function* 

This Is a disturbing state of affairs because It means, basically, 
that the Institution Is only interested In supporting and encouraging 
excellence In research* Its consequences are obvious In that faculty, 
regardless of their interest, will neglect their teaching and service 
activities In order to attain the professional recognition required to 
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remain and succised at their Infttltutlon* Students are perhaps the most 
unfortunate pawns In such a game^ since they are forced to take courses 
from faculty who are not able or willing to take the time to prepare and 
organize their courses or to do such things as spend tlnn outside of 
class discussing problems and concerns that vould help mofit students 
learn material better* 

Some take the position that there is no inconsietency in chis type 
of skewed reward syetem since it Is suggested that excellent researchers 
are* in fact» the best teachers* The research evidence (see Aleamoni» 
1974)» however* does not support this point and shows that» in f«Hisral» 
there is no relationship between scholarly prcHluctivity and effective 
teaching* 

It is further suggested that the evaluation of scholarly productivity 
la vmch easier » more valid* ana more reliable than is the evaluation of 
instructional effectiveness* This is probably the most critical point in 
the problem of rewarding faculty at institutions of higher education* It 
can best be exemplified by quoting from the November^ 1971» University of 
Illinois at Urbana^bampalgn pantphlec on P romotion and Tenure Folic lea of 
the College ^ Liberal Arts and Sciences beginning on page 4: 



'^Evaluation of teaching Is difficult* but evaluation based 
on systematically gathered data into bread categories of * excel* 
lent'v *good\ *average\ and 'poor* Is possible* Aiwng the 
most useful kinds of evaluative evidence are testimony of execu- 
tive officers* especially when based on student intervievs 
covering several semesters* coimnents of colleagues who are well 
acquainted with the teaching perfonnance of the candidate* 
achievement of students* and the quality of teaching utateriala 
prepared by the staff member. A significant element in the 
evaluation of teaching is the overall judgement of students* 
and the department is responsible for obtaining such information 
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on all staff members, particularly those recnomended for promotion. 
Questionnaires developed in cooperation with the Office of 
Instructional Re3ource5t may be used for this purpose, or a 
similar procedure can be followed which is designed to reflect 
comprehensive judgemenr. concerning teaching qualities. Faculty 
members i^se records conGisten:.ly reflect poo. teaching should 
normally not be rcconc^aded for promotion. 

In unusual circumstances, tenure may be recofmnended for 
deotonstrated excellence in teaching Aven in the absence of 
significant published research. ^Nevertheless, the Executive 
Committee believes that the qualifications for teaching and 
scholarship are very closely related, the teacher who does 
not keep current with developing knowledge in his field or 
who is not constantly searching for new insights does not have 
a future as a classroom teacher in the college. Graduate as 
well as undergraduate instruction is a responsibility of the 
faculty of this College; a continuir ; interest in, and a capa^ 
city for, creative scholarship by a faculty member is essential 
for effective lastructinn for undergraduate as well as graduate 
students. A Jiaculty member who lacks the qualifications to 
teach advanced students will cot be recoimaended for promotion 
to senior ranks. 

Unlike the assessment of teaching, an accurate judgemrat 
of scholarly achievement generally may be made best by the 
academic community at large. To be considered for promotion 
to an associate professorship, the candidate should have done 
work of sufficient quality to have gained recognition by his 
peers and standii in his profession. A person to be eon* 
sidered for a protessorship should be a scholar who has achieved 
national distinction. 

GTldence of favorable judgement by colleagues Includes 
publication in journals where expert evalitation is required 
for acceptance, favorable reviews of books, appointmeKts or 
awards that require evaluation of professional competence, 
election to office in learned societies, ^nd receipt of 
fellowships. 

Although departments have the first respons'^.riXlty for 
evaluating the quality of the work of a candidate for tenure 
or promotion, it is within the scope of the Executive Com- 
mittee's responsibility to gather confirming ev:'d''ace of 
scholarly competence by seeking the comments of othar scholars 
within and without the University.** 
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The quote presented on the previous page reflects the policies of 
most institutions of higher education. It 18 important to note that this 
policy indicates that, regardless of the quality of the eveliative teach- 
ing evidence: published refSf^arch will still take precedeace in the 
reward copsi-derations. It also states that the relationship between 
teaching and scholarship la believed to be high and positive even though, 
as was stated above, the research evidence to date does not support this 
belief. If the Institutions of higher education are really only inter- 
ested in rewarding research productivity, then they do not need large 
student enrollments nor do they need to offer a large number of courses. 
Instead, they should concentrate on kiaving their faculty do only research 
and admit small numb^^rs of graduate students who can work directly with 
the facult^ in their research luoject. . This means that teaching and 
service activities should ba relegated to undergraduate teaching inatl- 
'^Mtions. 

Now, let us look mi^re closely at the criteria used to judge scholarly 
achievement. In tht! above quote they rest basically on the belief that 
judgments by colleagues (synonomous with "the academic coamnmity at large**) 
provide the ixna^ evidence. These judgments include (a) publication In 
joui'nuls where expert evaluation is required for acceptance, <b) favorable 
reviews of books, (c) appointments or awaicds that require evaluation of 
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professional competence^ (d) p^'ectlmi to office In learned 3ocletltt5» and 
(e) receipt of fellowshJi^K* If one lookn closely the above judgmentst 
it Is obvious that each of thesis Is '*ery subjective; It^ fact« It would be 
very difficult to find a set of objective criteria on which ittany colleagues 
could agree when Judging Journal publications^ books« etc* Anyone who has 
read several advisory edlcors* reviews of their article or experienced the 
acceptance of a once rejected article by another reputable journal cou^d 
attest to this subjectivity* 

Where then is the rationale for maintaining that the evaluation of 
teaching baaed on (a) the testimony of executive officers^ (b) comments 
of colleagues who are well acquainted with the teaching performance of the 
Instructor t (c) achievement of students » (d) quality of teaching materials 
prepar^^ by the instructor^ and (e) the judgments of students ^Kposed to 
the Instructor 1 is more subjective^ leas reliable and less accurate than 
the judgments of scholarly achievement? Apparently^ the answer lies in 
the fact that the Toethods and techniques used in evaluating scholarly 
achleveme:it have ranalned unchanged for so long that they have become 
accepted as standard without quescion or evaluation* In contrast » the 
evaluation of teaching has been sub j tut to a wide variety of approaches 
with no commonly accepted methods ot techniques except for student judg- 
ments gathered via rating foTms» some of which have not« admittedly^ been 
professionally designed* 

If institutions of higher educ^atlon are serlou<^ about encouraging 
excellence in teachings then they muse be pwpared to reward such excel'* 
lence In a manner similar to that for excellence in scholarly achievement* 
The first step in achieving this goal would be to establish a standard 
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system of evaluating Instrttctionr^ effectiveness which would have as Its 
two major outcomes (a) reward tor success and <b) guidance for Improvement* 
The rest of hid paper will be devotii to outlining jusc such a system for 
rewarding and Improving Instructional effectiveness* 
Methods of Assessing Instructional Effectiveness 

Evaluation o£ student attitudes about Instructors and Instruction Is 
perhaps t^e most commonly used method of assessing Instructional effective* 
ness# The rationale for this is that students are assumed to be the only 
ones who are constantly exposed to the elements of the course (e*g*^ 
Instructor^ text book» homework^ course content^ method of Instruction^ 
etc«) sndf therefore^ the most logical evaluators of the quality and 
effectiveness of those elements* In addition^ student attitudes should 
Indicate areas of rapport, degrees of communlcatlcn, or the existence of 
problems and thereby help Instructors as well as Instructional resource 
personnel describe and define the learning environment more concretely and. 
objectively than they could through ouher types of measur^ents* Such 
attitudes can be ascertained simply by determining the number of students 
who agree or disagree with certain statements about the course* or by askr 
Ing students to write short essays about the course In order to obtain 
Informat^^on about their experiences under specific Instructional sltua- 
tlons# Such Indivlduall^sed procedures do not» however* provide an 
opportunity to compare the results of one course with results ot another* 
Measurement Is more useful when comparative results are available* Thus 
more adequate Interpretation will occur (a) when the data has been col* 
lected In a s^iandardl^ed fashion with appropriate attention given to 



ERLC 



7 



BESi COPY AVMLE 

7. 

san^llngf reliabllltyv and validity; and (b) when many instructors and 
instructional programs have been measured with the same instrument so that 
comparisons ran be made (Aleamoni an^ Spencer ^ 1973). An additional 
source of ueteful information might be ratings by graduating seniors of 
their luatructor^ and Instruction <i)rttcker and Remmers* 1950:. 1951; Alea- 
moni and Timer, 1974) . 

Faculty (peer) and department head attitudes and judgments represent 
another method of assessment. These should be based upon an evaluation of 
(a) the procedural adequacy of the course(s); <b) the appropriateness of 
the course content; <c) the appropriateness o£ tl*^ courbe objectives; <d) 
the adequacy and quality of the course examinations; (e) the instructor's 
self-report; and (f) other course materials ot conmients which the instruc- 
tcr wishes to submit* Such evaluations should be made by departmental 
review commititees with well-structured guidelines and criteria of excellence. 

The quality of student learning is another very important variable to 
consider. It would be possible to get a measure of student achievement in 
a course by employing departmental or agreed upon nationally standardized 
examinations xa the subject matter area. These could be used as final 
examinations or as part of the final examination. Other measures could ^e 
obtained by follow-up measurements of student performance in succeeding 
courses or by following the students after they have left the imiversity 
am recording their performance at other institutions of higher education 
or in their Jobs or professions. These mttasures should then be evaluated 
by a departmental review committee utilizing instructional resource per- 
sonnel where needed • 
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Feedback t o the In structor 

The student attitude or rating daca should be presented to the 
Instructor 1. an easily readable and Xnterpretable formut* Student 
responses to the questionnaire Itams should be reported in ^ussoaty form 
for each course and appropriate normative ccinparisons should be noacle 
between the mean scores In the instructor's course (by each Item and 
groups of items) and meann for similar Instructors teaching similar 
courses In the same department^ college^ and university* For other exam- 
ples of appropriate normative comparisons^ aea Costing Greenough^ and 
Manges (1971) and Aleamoni (1972, 1974)* 

The instructor should be provided with additional Interpretation 
loateriaX as well as with the possibility of consultation vlth the 
instructional resource personnel responsible for providing the student 
attltudlnal information* Any additional evaluative Information and 
resources needed in this area should also be dlt^cussed with cha instruc** 
tlonaX resource personnel* All interpretation material and consultation 
afaould concentrate on identifying instructional strengths and weaknesses 
and then agreeing on what should be done to remedy the we'tknesses* 

Faculty (peer) and department head attitudes and j^Jdgments should be 
presented to the faculty member in an easily readable and Intarpretable 
report* This report should outline the committee and individual judg* 
ments made on each element of the instructional se;:tins considered and 
identify the strengths and weaknesses observe* Constructive criticisms 
that would allow the Instructor to plan a strategy for Improvement of 
the areas of weakness should appear In the report* Per:?onal consultation 
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dhould take place between the department head and the faculty member con 
cernlng all aspects of this report* The departi&ent head should expXain 
the options or resources available to the instructor to belp him in his 
attMtpt to Impx^ove his instructional weaknesses* 

The quality of student learning data gathered should also be pre- 
dated to the instructor in an easily readable and interpretable report 
Sludlar to the one for faculty (peer) and department head attitude^s and 
judgments* This report* however* should concentrate on the achievement 
of coarse« department, college* and university instructional objectives* 

The department head should hold individual conferences with ins true 
tors concerning all Che evaluations that have been conducted and what 
they mean to the instructor* (This, of course^ assumes that the depart- 
ment head understands the evaluations made*) The deptartment head should 
be emphasizing what the instructor needs to do in order to rectify the 
weaknesses or problems uncovered in the various evaluations made* The 
department head should be clarifying how such evaluations and efforts 
will be used in the rank* pay, and tenure considerations of the Individ*^ 
ual faculty member* 
Qptluns for the Instructor 

If all Che assessments of instructional effectiveness are highly 
positive, then no more evaluations or data are needed* All the repox*ts» 
aCc*» of the evaluations should be placed in the instructor's depart- 
mental file to be used for rank» pay» ar^ tenure considerations* 

If part c^r all of the assessments o^ instructional effectiveness 
are not highly positive^ then the instructor should determine (with the 
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possible aid of instructional resource persosnel as well as the department 
head) what needs to be done to io^rove the instructloaal weaknesses, these 
needs could then be^ presented in the form of a propoaal generated in the 
following manner* 

1* the instructor should draft a proposal outlining vhat needs to be 
done, what instructional resources will be needed, how long it will take, 
how much it will co8t> and how it will be evaluated. 

2. He should discuss his proposal with his department head and 
obtain any necessary approval (if needed) and additional resourcts <if 
needed). 

3. Be also should discuss hie proposal with the appropriate instruc- 
tional resource personnel to detertiine what services are available to htffl 
in trying to impleanent the proposal. 

4. Finaily> the instructor should modify the proposal according to 
the suggestions made in his discussions with the department and instruct 
tional resource personnel. 

Imp lemfen ta t ion of, the Instructional Tmprovement 
Proposal and the Pro.^ect*8 Final Repor t 

Once a final draft of the proposal is generated and all approvals 
obtained and resources identified* the instructor should initiate the 
timetable needed to implement his proposal. 

After the instructional Improvement project has been completed and 
all the evaluation has been done> thr Instruccor should prepare a compre- 
hensive final report. Instructional resource personnel could be used at 
this point iz needed. t\%t, final report and the proposal should then be 
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snbiiiitted to the department head and placed in the .atructot's departmental 
file for use In rank, pay, and tenure jn^lderations* 

Admlnlstr tlve Procedutg for ConsiderlnB^ Instruction in the Ingtitutional 
Reward Systen 

If all of the instructional effectiveness assessments were highly posi- 
tive, then that evidence should be considered relative to the teaching load 
required of the faculty member* That evidence should be weighted appropri- 
ately in a "teaching -4- research + service** equation in arriving at rank, pay, 
and tenure recommendations* 

If lan instructional Improvement proposal and final report ate in the 
instructor's departmental file, then they should be carefully analys'^ and 
evaluated by a departmental review committee utilizing methods and techniques 
similar to those in evaluating scholarly achiev^oent* This evidence ehould 
be considered proportionate to <a) the teaching load required of the faculty 
m^ber and <b) the time needed to generate the proposal, complete the pio* 
ject, an^ write up the final teport* That evidence should be flighted 
accordingly in a "teaching improvement effort + teaching + research + service" 
equation in '•rrivirg at rank, pay, and tenure recommendntions* 

In conclusion, the higher level administration of both the institution 
and the faculty should accept and encourage the use of such a aystem as an 
integral part of the institutional reward scheme* Not to do so betrays a 
lack of interest in and a lack of concern with instruction and instructional 
improvement* 

It should be noted chat some of the methods of assessing instructional 
effectiveness ate already being used in a few institutions of hlghet education 
in the United States* 

Although the service component was not treated in this paper, a similar 
evaluation system should be developed for ic* 
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